Development and Validation of the Adolescent Sexual and Reproductive Competency Assessment Tool (ASRH-CAT) for Healthcare Providers

Objectives: This study aimed to validate a competency assessment tool for adolescent sexual and reproductive health (ASRH) services for healthcare providers (HCP) at primary healthcare (PHC) facilities that require a specific set of competency skills to address ASRH problems. Methods: The tool development process used the nine steps of scale development and validation. Fifty-four items were yielded through the expert panel discussion. Two hundred and forty respondents were recruited for an online questionnaire using non-probability sampling. The item content validity index (I-CVI) and exploratory factor analysis (EFA) were used for construct validity. Results: Fourteen items were removed based on the I-CVI (scores < 0.8) and two items were removed in the EFA (factor loadings < 0.4). The reliability analysis, according to the latent factor, yielded a good item-total correlation (ITC) and a good internal consistency value, with Cronbach’s alpha values of 0.905–0.949. Conclusions: The final ASRH competency assessment tool (ASRH_CAT) contains 40 items and is reliable and suitable for use in studies related to the ASRH competency assessment of HCPs at the PHC level.


Introduction
Globally, sexual and reproductive health (SRH) is a leading public health problem among the early-reproductive-age group, which requires appropriate services for preventive measures [1,2]. Adolescent SRH (ASRH) problems are associated with a lack of SRH knowledge; socio-cultural norms regarding sexual activity; healthcare workers' negative behaviors; and the awareness, availability, cost, and quality of the SRH services provided [1][2][3][4][5][6]. Women face higher SRH risk, as their health status determines their potential offspring, but there is scarce evidence that they have been targeted in the literature, especially among young adults [1,3]. Unmet SRH needs are always emphasized in adolescent health intervention, but the literature measuring healthcare providers' (HCP) competency skills in delivering SRH services is scarce [6][7][8]. Little is known about healthcare workers' management of personalized care for adolescents regarding SRH and their competency regarding risk screening, decision-making, and creating a shared care plan. The Countdown 2030: Drivers Technical Working Group highlighted that SRH is the key aspect of the disease burden of adolescents, for which challenges should be overcome [8]. The provision of adolescent-friendly health services by HCP is listed as a core health sector mandate that requires service delivery monitoring. The ability of HCP to advocate healthy lifestyles to promote good SRH screening and identify those in need of treatment and follow-up is crucial. Investing in dedicated, competent staff will foster the management of adolescent service usage and reduce service inequalities. A competent HCP will ensure the quality of care, establish a relationship with the adolescent, and promote open communication [9]. Concerns regarding the disclosure of diagnosis and treatment require the HCP to undergo a certification training program to address adolescent needs. The challenges include providing SRH and managing adolescent cases with an understanding of the local cultural and developmental stage of adulthood from the identity stage to the role of confusion and from the intimacy stage to isolation and full independence [9]. The National Research Council and Institute of Medicine report on adolescent health services (2009) stated that most HCPs working at PHC are nurse practitioners via adolescent health-friendly clinics and school health services led by family physicians or medical doctors. Therefore, the present study aimed to develop and validate an ASRH competency assessment tool (ASRH_CAT) tailored to the local socio-cultural needs and diverse HCP levels that serve at the PHC level.

Study Design and Settings
The present validation study was aimed at developing a novel ASRH competency questionnaire. The study was conducted over 12 weeks from June 2021 to August 2021, which coincided with the fourth wave of COVID-19 in Malaysia [20]. Therefore, respondents were invited by various Malaysian HCPs, who were attached to PHCs, to participate in an online survey. The population-based survey was carried out during the Movement Control Order (MCO) of the fourth wave of COVID-19 when Malaysia went into strict lockdown. Various COVID-19 pandemic management activities, including COVID-19 screening at the COVID-19 assessment centers (CAC), isolation and treatment at the COVID-19 quarantine and treatment centers (PKRC), and the national COVID-19 immunization program (PICK) were ongoing at the public PHCs and involved many HCPs. Therefore, all respondents were recruited using an online platform via email and WhatsApp blasts linked to a Google Form. The online platform was chosen to reduce the risk of in-person contact that would have been vulnerable to droplet transmission of the COVID-19 virus. The coverage of internet access and digital media among the HCPs, as well as many healthcare programs delivered during the pandemic, were planned using a digital appointment system.

Questionnaire Development
The ASRH competency skills assessment tool (ASRH_CAT) was developed in this study as a new assessment tool for assessing basic HCP competency in decision-making and planning ASRH services at PHCs. A self-administered, online, bilingual (English and Malay) survey was conducted involving healthcare practitioners, which included doctors, nurses, and medical assistants working at public PHCs. The questionnaire development was divided into three phases adapted from the best practices for developing and validating scales for health, social, and behavioral research [21].

Phase 1: Item development
Step 1: Domain identification and item generation We generated domains and identified items using literature evidence. A team of five experts (one public health specialist, two family health specialists, one obstetrics and gynecology specialist, and one adolescent psychiatrist) was appointed. All experts were contacted directly, and an in-depth interview using video calling was conducted with each expert using semi-structured questions to determine the domains for the new tool. Items were mainly generated using a deductive method focusing on the core competencies needed.
To construct the tool, the items were developed based on five guidelines regarding core competencies and decision-making for ASRH management in PHCs [13,17,19,22,23]. Later, the tool was screened for repetitiveness, complexity, and irrelevance before being included in the items pool. Four domains were determined to have the most agreement among all five experts: (1) ability to provide ASRH education (HE), (2) self-perceived capability (C), (3) self-perceived knowledge level (K), and (4) self-perceived attitude (A).
The questionnaire was drafted in English and then translated into Malay using forward and backward methods [24,25]. The translation involved two linguists fluent in Malay and English.
Step 2: Content validity An evaluation, completed by the experts and target population, was performed. The experts evaluated each item constituting the domain for content relevance, representativeness, clarity, and consistency to determine the item content validity index (I-CVI). Fifteen people from a targeted population group participated in evaluating each item constituting the domain based on their life experience handling the adolescent group.
The initial draft included 54 items, and 14 items were removed based on the expert consensus agreement due to irrelevance, redundancy, and unclear statements. The remaining 40 items were scrutinized and rephrased to eliminate complexity, as suggested by the expert group. The final 40 items were able to measure the basic competency needed by HCP according to the Ministry of Health Malaysia guidelines [13,17,19,22,23] for implementing ASRH services at PHCs.

Phase 2: Scale development (construct validity)
Step 3: Pre-testing questions (cognitive interviews) A total of 40 items were included in the items pool. All items were constructed in the form of statements and the respondents rated their self-perceived competency level using a 5-point Likert scale (0 = strongly not confident and 4 = strongly confident). Subjective responses during the answering process were reflected through their comments.
Step 4: Survey administration and sample size To estimate the sample size for the validation study, assumed indiscriminately, each item used was assessed by 5-10 people. The Kaiser-Meyer-Olkin (KMO) sampling adequacy test was used to ensure an adequate sample size [26,27]. Therefore, this study targeted a minimum of 200 samples, and we managed to recruit 240 respondents. A cross-sectional study was conducted for exploratory factor analysis using state healthcare workers serving PHC facilities under the Kedah State Department of the Ministry of Health of Malaysia. Kedah state is located in northern Malaysia and has rich socio-cultural practices that influence lifestyle behaviors in society.
Step 5: Item reduction The proportion of items with complete responses was determined. According to Moret et al. [27], the psychometric analysis of the scale can be optimized when items with many missing responses are deleted to ensure the availability of complete cases for scale development and improve the item response distribution by reducing the ceiling effect. This is the first questionnaire that measures healthcare workers' perceptions of their competency in managing SRH. Therefore, we planned to conduct a subsequent study using a diverse population when the pilot study had been completed to assess the questionnaire's validity. During scale development (content validity), we omitted items with responses that were not available or applicable.
Step 6: Extraction of factors: Data An exploratory factor analysis (EFA) was conducted to determine the optimal number of factors or domains that fit a set of items.

Phase 3: Scale evaluation
Step 7: Tests of dimensionality We created scale scores to allow for reliability and validity analysis. The ASRH_CAT has 40 items and 4 domains with a range in scores of 0c100. The scale scores were calculated, and the mean of the raw item scores was computed. Each domain is calculated independently. A higher score indicates a better competency level. The calculation for each domain is as follows: Aggregation scores for all domains are not calculated, as the competency level needs to be assessed based on the domains.
Step 8: Reliability testing We conducted test-retest reliability at 2-week intervals with the same set of respondents to establish whether the responses were consistent when repeated. The internal consistency of the scale was estimated with Cronbach's alpha [28].
Step 9: Test of validity We assessed the feasibility (relevancy, representativeness, clarity) of the questionnaire to be used (face validity). We recruited six respondents (not involved in the real study) and used a scale of 1-4, where 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = highly relevant for each item. We analyzed the test using Cohen's kappa index for inter-rater agreement and used the results to guide questionnaire improvement.
We conducted a pre-test of 20 healthcare workers who were not involved in the recruitment of the true study. The validity assessment of the questionnaire was performed using the difficulty index and the item discrimination index. First, the validity of the questionnaire was assessed based on the proportion of respondents who answered the items correctly (item difficulty index). A higher value for the item difficulty index indicated that more respondents were practicing competently. The item discrimination index measured how well an item could differentiate between respondents who were competent vs. not competent in managing ASRH cases.
This set of questionnaires is the first competency tool available to assess the selfperceived competency level in healthcare providers' management of ASRH care. Therefore, no published, validated study instrument can be used as a benchmark to assess concurrent validity for the same purpose.

Statistical Analysis
Statistical analysis was performed using SPSS v.26. All 40 items were computed in the analysis. Varimax oblique rotation was used for the EFA measurement. The number of factors retained was determined using the Kaiser criterion with eigenvalues > 1. Items were suppressed when the factor loading was <0.4. The Kaiser-Meyer-Olkin (KMO) index was used for sample adequacy, with a significant Bartlett's test for sphericity valuation. A reliability analysis was performed, and Cronbach's alpha was determined.

Content Validity
The expert panel discussion yielded a total of 54 items, as presented in Table 1. The questionnaire contained four domains: (1) self-perceived ability in ASRH education (HE), (2) self-perceived capability in ASRH education (C), (3) self-perceived knowledge (K), and (4) self-perceived attitude (A), with 13, 20, 11, and 10 items, respectively. For the domain of self-perceived ability in ASRH education, the highest mean score was for item HE7 (3.76, SD 0.909). For the domain of self-perceived capability in ASRH education, items C9, C10, and C11 had the highest mean score of 3.70. For the domain self-perceived knowledge, item K7 had the highest mean score of 3.70 (SD 0.825). For the domain self-perceived attitude, item A5 yielded the highest mean score of 2.90 (SD 0.850).

Construct Validity (EFA)
A cross-sectional study was conducted among the 240 healthcare workers recruited from Kedah State ( Table 2). Most of the respondents were Malay women, and the mean age was 39.6 years (SD 7.1). More than half of the respondents possessed at least a diploma and worked as a nurse. The mean length of service in the public health sector was 12.46 years (SD 6.50). A cross-sectional study was conducted for exploratory factor analysis. The KMO index for the questionnaire was 0.94, which indicated that the sample size employed in the validation study was sufficient to run the analysis. The p-value for Bartlett's test of sphericity was significant at p < 0.001, signifying the presence of multidimensionality in all the items.
The EFA revealed that four latent factors were detected, as shown in Table 3. The reliability analysis, according to the latent factor, yielded a good item-total correlation (ITC) and a good internal consistency value, with Cronbach's alpha values of 0.905-0.949. Only two items from the domain self-perceived ability to provide ASRH education (HE8 and HE11) were removed due to low factor loadings (<0.40).

Reliability Testing
During the pre-test study, we conducted test-retest reliability at 2-week intervals that involved six healthcare workers who were involved in SRH services to establish whether responses were consistent when repeated. The internal consistency of the scale was estimated with Cronbach's alpha. The results obtained ranged from 0.8 to 0.9 for each item.

Face Validity
We assessed the feasibility of the questionnaire based on the items' relevancy to the domains, the items' representativeness of the study objectives, the items' relevancy to the concepts related to the study topic, and the clarity of words or terms used (face validity). During the pre-test study, we recruited six healthcare workers who were involved in SRH services and used a scale of 1-4: 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = highly relevant. We analyzed the results using Cohen's kappa statistic for inter-rater agreement. Cohen's Kappa for relevancy to domains was 0.9, representatives to objectives were 0.8, relevancy to study topic was 0.7, and clarity was 0.8. The format is user-friendly and takes less than 10 min to complete. It is self-explanatory and can be administered via an online survey or manually using paper.

Item Difficulty Index
We conducted a pre-test among 20 healthcare workers who were not involved in the recruitment for the true study. Item analysis was conducted for the question items in each domain. Subsequently, respondents with a score of 50% of correct items within the domain were assigned a 1 for a correct domain, while respondents with a score of <50% were assigned 0 (for an incorrect score for the domain). The item for the domain difficulty index was calculated and ranged from 0.4 to 0.7 (Table 4). Therefore, no domain was omitted from the questionnaire. No items were omitted from any of the domains, as the item difficulty index > 0.6. The item discrimination index was calculated and ranged from 0.4 to 0.6; therefore, the domain was considered appropriate to be used.

Discussion
The WHO created the core competencies manual to assist HCPs in guiding adolescents in receiving quality SRH services regardless of their background. However, due to sociocultural diversity, some countries may need to tailor the competency assessment according to their socio-cultural background, local policies, and HCP training availability. The Adolescent Sexual and Reproductive Competency Assessment Tool (ASRH-CAT) is a self-administered questionnaire for HCPs that was newly designed explicitly for ASRH services at Malaysian PHCs. The tool was developed based on WHO guidelines, and the services delivered at adolescent health clinics were tailored to local guidelines. Therefore, the tool might be applicable to be used by any country interested in assessing HCP competency levels before enrolling them in specific SRH courses or placing them in the ASRH services to cater to adolescent health. The ASRH-CAT can be used to assess HCP readiness to be in charge of the adolescent health program. The questionnaire items were composed based on five different ASRH core competency guidelines. Each guideline varies in terms of the presentation and arrangement of the competencies by domains, thereby creating numerous core competency items that included thorough individual requirements for the measurement of competency skills. A total of 54 items were chosen and pooled according to the domains identified from the local expert consensus and agreement based on the Ministry of Health of Malaysia ASRH guidelines [23]. Each item was designed to measure knowledge, opinions, and attitudes in managing ASRH using a Likert scale from 0 (strongly not confident) to 4 (strongly confident). The items of each domain were grouped to encourage respondents to focus and allow them to select the most appropriate answer to indicate their positive strength of agreement or feeling of connection to the services delivered. All items were in positive phrase order to avoid confusion. Grouping item responses by adding them demonstrated more reliable measurement. Scores range from 0 to 100, with a score of 0 indicating an individual was not competent and 100 indicating competence in ASRH. Higher scores indicated better knowledge, attitudes, and opinions on the four domains: Domain 1: self-perceived ability to give ASRH education (HE = 13 items); Domain 2: self-perceived capability in ASRH management (C = 11 items); Domain 3: self-perceived adequate ASRH knowledge in decision-making (K = 11 items), and Domain 4: self-perceived appropriate ASRH management attitude (A = 5 items).
Currently, many SRH courses are available online as free training or with a minimum course fee to enroll. Most of the courses are provided through universities in non-Muslim countries. Malaysia is very ethnically diverse, with Islam being the main religion. Teaching ASRH is emphasized in Islamic practice. However, it is not implemented well at school, as it is considered a sensitive subject and teachers are unprepared to teach the SRH topic due to insufficient knowledge [28]. The religious scholars who normally teach pupils attended Islamic schools, did not receive any formal structured SRH training for children. Meanwhile, parents teach SRH to children in a limited scope based on convenience, life experience, and understanding. The poor shared knowledge of SRH risk prevention among adolescents will result in health problems because of not being alert to the risk of danger and the tendency to delay seeking medical or health assistance. Tailoring ASRH education based on local needs, as reported in public health evidence, is highly important. The HCPs at PHCs are at the forefront of health screening. Therefore, to overcome the issue of inadequate ASRH education, it is essential that professional PHC training be assessed based on a validated tool that can provide a structured framework for addressing ASRH needs.
All of the domains in the questionnaire demonstrated Cronbach's alphas > 0.9, which is considered high and may suggest high redundancy, whereas some items might test the same question under different wording. The high Cronbach's alpha demonstrates that the test should be shortened. The evidence revealed that the acceptable alpha values should fall between 0.70 and 0.95, with a recommended maximum alpha value of 0.90 [29]. However, we opted to retain the remaining items due to the need to measure different subjects relevant to ASRH core competencies. Removing some items may cause the tool to lose its ability to assess fundamental competencies, as many ASRH services and problems are involved. The assessment tool should measure the essential competencies needed to provide quality SRH services to adolescent clients at PHCs, especially in decision-making and planning.
The high number of items in the tool initially raised concerns among the participants due to the amount of time needed to complete the assessment. However, the face validity assessment showed that the ASRH_CAT is user-friendly, self-administered, and self-explanatory and can be completed within less than 10 min. It also can be incorporated with other study variables and parameters, such as sociodemographic variables, working and training experiences, and practice in ASRH service, to assess associations with the level of competency.
The questionnaire was constructed and followed the content validation stages [21]. It focuses on decision-making and management planning while recognizing the core competencies of ASRH service in PHCs and the factors that may affect competencies, such as training experience, work experience, education level, and job title. From this study, some essential factors regarding the unanimity of the instrument scores tested within the target population need to be inferred. First, the high level of agreement may be because the respondents recognized that ASRH features essential competencies common to HCPs at PHCs across geographic locations, such as a national and international standardized approach for developing and implementing competencies [30]. Second, the sample mainly included nurses (77%) who had an average of 12.5 years of experience working at PHCs. Only 7% of the respondents had been assigned to adolescent health clinics, and approximately 10% had received at least one formal ASRH training. They were more likely to report moderate to high competencies for the ASRH service, as they may learn the skills through observation and informal training [4].
It is essential to acknowledge that no single assessment model can evaluate all competencies and that different models may measure similar competencies differently and with different levels of precision given their measurement properties. Ideally, the optimum plan will identify multiple assessment models by combining low-and high-fidelity approaches relevant to the competencies to be measured and considering the different stages in professional development and practice. Where possible, care should be taken to avoid one-shot testing, through reliance on a single assessment model, in making critical decisions at any stage of professional development [31].
This tool has its limitations, in that it may not fully cover all the competencies and skills warranted for the HCPs at PHCs based on WHO guidelines, as such assessment tools may be lengthy and time-consuming. Nevertheless, this tool is sufficient for assessing the essential ASRH competencies in decision-making and planning in Malaysian PHC settings for self-competency monitoring. This tool is not meant to be used as a single-measurement tool to assess competency. Repeated assessment may be needed after some exposure at work, following training, or even annually, as deemed appropriate by the organization [32]. This study's limitation is that this is the first version of the tool validated among the HCPs working in the PHCs serving the Kedah state population alone, and most participants were nurses. Considering the current situation at PHCs during the COVID-19 pandemic, which has caused HCPs to work overtime and burn out from the heavy workload, such feedback from the participants is valuable.
Ground-level implementation is anticipated for this tool to be used to assist HCPs and administrations in choosing HCPs for training or courses. The advantage of this tool is that it can be implemented as a new approach to assess the HCP's competency before and after the training instead of using the participant's satisfaction rating of the training itself, as previously conducted. Additionally, this tool was developed based on the targeted population at the primary healthcare level and contains items appropriately validated according to the training modules for assessing the ASRH competency of HCPs at PHCs. Even though the validation process was conducted during the COVID-19 pandemic, the HCPs are capable of performing their work according to their competency requirements [33]. The ASRH-CAT is planned to be used as a baseline assessment of any adolescent health program, to monitor ASRH staff competency levels, and to plan training courses. Future studies with larger samples from the general population are recommended to test its psychometric properties to improve reliability tests and compare with our tool (Supplementary file-final ASRH_CAT questionnaire). Other countries can use the questionnaire and validate it in their population after it is translated into their own language.

Conclusions
The ASRH Competency Assessment tool (ASRH_CAT) was designed to identify ASRH competencies among HCPs in Malaysian PHCs and exhibited satisfactory content validity. This study presents a significant contribution by allowing for the measurement of the competencies through a self-administered instrument. It can be used as the first step in identifying strengths and gaps in knowledge, opinions, and attitudes of ASRH care, thus aiding in future strategic planning for care quality and training plans.

Institutional Review Board Statement:
The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of UKM Research Ethics Committee (protocol code FF-2020-423 and date of approval 5 October 2020).

Informed Consent Statement:
Informed consent was obtained from all subjects involved in the study.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.